I mean—I hear what you’re saying, and I don’t agree. It’s not that important of a post, so it’s fine, but I think it doesn’t really hook people.
I don’t think it communicates what the post is about “We Added Typos to a Benchmark. Then Haiku Saturated It.”
I don’t think it’s about saturation at all—as evidence of this, I ran “ctrl-f” for satur and there are 0 mentions to it in the body.
less critically—“we added typos to a benchmark”—this is more of the action than the reason, which kind of just leaves someone asking “why is this relevant”
Got it. I was indeed thinking to frame this blogpost as more lightheartedly. Would this be better: “We added typos to a benchmark, then Haiku’s scores jumped” so there is no mention of “saturation”. I’m thinking the blogpost is more of “why did Haiku’s score jump” instead of “LLMs are robust to typos”.
I mean—I hear what you’re saying, and I don’t agree. It’s not that important of a post, so it’s fine, but I think it doesn’t really hook people.
I don’t think it communicates what the post is about “We Added Typos to a Benchmark. Then Haiku Saturated It.”
I don’t think it’s about saturation at all—as evidence of this, I ran “ctrl-f” for
saturand there are 0 mentions to it in the body.less critically—“we added typos to a benchmark”—this is more of the action than the reason, which kind of just leaves someone asking “why is this relevant”
Got it. I was indeed thinking to frame this blogpost as more lightheartedly. Would this be better: “We added typos to a benchmark, then Haiku’s scores jumped” so there is no mention of “saturation”. I’m thinking the blogpost is more of “why did Haiku’s score jump” instead of “LLMs are robust to typos”.